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BACKGROUND OF THE INVENTION 

Field of the Invention 
The present invention generally relates to a method and system to 
embed an image. More particularly, the present invention relates to a method 
and system to embed one or more images into two or more other images. 

Description of the Related Art 

Recently, several watermarking schemes have been proposed where a 
watermark is embedded in two halftone images such that the watermark can 
be extracted by simply overlaying these halftone images. 

In one conventional method, the two halftone images are created 
using dither masks, while another conventional method creates the two 
halftone images using an error diffiision technique. The watermark images in 
these schemes are binary images and any correlation between the binary 
pixels of the two halftone images depends upon whether corresponding pixels 
in the watermark are black or white. 

There are several limitations to these conventional methods of 
extracting watermarks. First, the watermark images must be binary and 
cannot be grayscale or color. 

Second, because the watermark is embedded in the two images based 



upon a correlation, the average intensity of a region of pixels is needed to 
identify whether the watermark pixel is black or white. Therefore, the 
watermark images that may be extracted using these conventional methods 
may only be simple graphics. In other words, the watermarks are limited to 
simple images such as logos or simple graphics and cannot contain detailed 
features. 

Third, the watermark images that are extracted using these 
conventional methods contain residual patterns and features from the two 
halftone images in which the watermark was embedded. These residual 
patterns and features reduce the fidelity of the watermark image that is 
extracted. Therefore, this is another reason why a watermark image cannot 
have fine details and be successfully processed using a conventional 
watermarking method. 

Fourth, the embedding by these conventional methods is only capable 
of generating binary halftone images, rather than grayscale or multi-bit 
images. These binary halftone images also limit the fidelity of the extracted 
watermark. 

Consider an image as a matrix of pixels, i.e. an image G includes 
pixels G(i, j) where G(i, j) , the (i, j)- th pixel of image G, is a vector in a color 
space. For grayscale pixels, the color space may be one-dimensional whereas 
for RGB pixels, the color space may be 3-dimensional. A set of n images Gi, 
G2, . . Gn of the same size can be considered as a matrix of n-tuples of pixels. 
This is denoted as Gi x G2 x . . . x Gn, i.e. the (i, j)-th element of Gi x G2 x . . , 
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X Gn is the n-tuple (Gi(i, j), . . . , Gn(i, j)). Equivalently, a set of n images can 
be considered as a single image whose color space is the Cartesian product of 
the n color spaces of the images Gi, G2, . . Gn. 

Consider an image transform "O" which acts on an n-tuple of pixels 
and produces an m-tuple of pixels, i.e. O (pi, . . . , pn) = (qi, • • • , qm) where pi 
and qi are pixels. By applying the transform O to each of the n-tuples of 
pixels in Gi X G2 X . . . X Gn, this transform acts on a set of n images and 
produces m images. The m auxiliary images may be watermark images that 
are extracted by the watermark extraction transform O. 

Another aspect to consider is how images are perceived under different 
viewing conditions. In other words, the n images plus the m extracted images 
can be considered as how a single image is perceived under different viewing 
conditions. Such an interpretation for the case n=l can be found in C. W. Wu 
et al, "Multiple images viewable on twisted-nematic mode liquid-crystal 
displays," IEEE Signal Processing Letters, vol. 10, no. 8, pp. 225-227, 2003. 

If a goal is to produce a set of n images, such that these images plus 
the additional m images that are generated by the transform O matches 
another set of n + m predefined images, perfect matching is not possible 
because there are more sets of n + m images than there are sets of n images. 

Instead of perfect matching, conventional watermarking methods 
utilizing halftoning take advantage of the fact that the images only need to 
look similar when viewed at an appropriate distance. Because of the "low 
pass behavior" of the human vision system, only low-pass versions of the 
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images need to match. In this manner, the human visual system reduces the 
amount of information in images, which allows the use of a digital half-toning 
algorithm to provide a solution to the problem. 

Yet another conventional watermarking scheme constructs two binary 
halftone images A and B, which when overlaid on top of each other reveals a 
watermark image C. Assuming that each pixel of image A and image B are 
either 0 (denoting a white pixel) or 1 (denoting a black pixel), the overlay 
operation can be expressed as C(i, j) = A(i, j) 0 B(i, j) where function "0" is 
the OR operation. 

These conventional methods embed the watermark image based on a 
correlation of the pixels between the two images and the ability to extract a 
watermark is based upon whether corresponding pixels in each of the two 
images vary together. For example, for a watermark image W that is a binary 
image, when W(i, j) = 0, the corresponding pixels in A and B are correlated 
and when W(i, j) =^ 0 the corresponding pixels in A and B are not correlated. 
This implies that when W(i, j) = 1 the overlaid pixels C(i, j) will be darker on 
average than when W(i, j) = 0 and, thus, C will reveal the watermark image 
W. However, as explained above, the pixels that are not correlated are darker 
than the correlated pixels only when averaged over a region of pixels. 
Therefore, the watermark cannot contain fine details. Furthermore, the pixels 
in the watermark image still reveal residual features of images A and B when 
overlaid. 
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SUMMARY OF THE INVENTION 

In view of the foregoing and other exemplary problems, drawbacks, 
and disadvantages of the conventional methods and structures, an exemplary 
feature of the present invention is to provide a method and structure in which 
an image is embedded into two other images. 

In a first exemplary aspect of the present invention, a method of 
embedding an image into two images includes performing a digital halftoning 
process on the Cartesian product of color spaces to embed the image into the 
two images. 

In a second exemplary aspect of the present invention, a method of 
extracting an image from two images includes extracting the image from the 
two images using a binary operation on each pair of pixels from the two 
images. 

In a third exemplary aspect of the present invention, a method of 
embedding a color image into two color images includes decomposing the 
color images into separate images in their color planes, and for each color 
plane, performing a digital halftoning process on the Cartesian product of 
pixel value spaces to embed the image into the two images, and combming the 
halftone images of the color planes into a single color image. 

In a fourth exemplary aspect of the present invention, a method of 
embedding a multi-bit image into two multi-bit images includes performing a 
digital halftoning process on the Cartesian product of color spaces to embed 
the multi-bit image into the two multi-bit images. 
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In a fifth exemplary aspect of the present invention, a signal-bearing 
medium tangibly embodying a program of machine-readable instructions 
executable by a digital processor. The program includes instructions for 
performing a digital halftoning process on the Cartesian product of color 
spaces to embed the image into the two images. 

In a sixth exemplary aspect of the present invention, a system for 
embedding an image into two images includes means for providing an image 
to be embedded into two images and means for performing a digital halftoning 
process on the Cartesian product of color spaces to embed the image into the 
two images. 

In a seventh exemplary aspect of the present invention a system for 
embedding an image into two images includes an image input device, and a 
digital halftoning device that performs a digital halftoning process on a 
Cartesian product of color spaces to embed the image received by the image 
input device into the two images. 

An exemplary embodiment of the present invention embeds a 
grayscale image into two images that have a substantially similar level of 
detail as the image being embedded. 

Additionally, an exemplary embodiment of the invention ensures that 
there is no residual image on the extracted watermark, thereby enabling 
extraction of a very high fidelity watermark. 

These and many other advantages may be achieved with the present 
invention. 



YOR920030609US1 



BRIEF DESCRIPTION OF THE DRAWINGS 

The foregoing and other exemplary purposes, aspects and advantages 
will be better understood from the following detailed description of an 
exemplary embodiment of the invention with reference to the drawings, in 
which: 

Figures 1 A - IC illustrate the results of an exemplary embedding 
method of one image into two other images in accordance with the present 
invention; 

Figures 2A - 2C illustrate the results of another exemplary embedding 
method of another image into two other images in accordance with the present 
invention; 

Figures 3A - 3C illustrate the results of yet another exemplary 
embedding method of another image into two other images in accordance with 
the present invention; 

Figures 4A - 4C illustrate the results of a further exemplary 
embedding method of another image into two other images in accordance with 
the present invention; 

Figures 5 A - 5E illustrate the results of a further exemplary 
embedding method of two images into three other images in accordance with 
, the present invention; 

Figure 6 illustrates a flowchart of an exemplary control routine in 
accordance with the present invention; 
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Figure 7 illustrates an exemplary image embedding system 700 for 
incorporating the present invention therein; 

Figure 8 illustrates an exemplary signal bearing mediimi 800 (e.g., 
storage medium) for storing steps of a program of a method according to the 
present invention; and 

Figure 9 illustrates another exemplary image embedding system 900 
for incorporating the present invention. 

DETAILED DESCRIPTION OF EXEMPLARY 
EMBODIMENTS OF THE INVENTION 

Referring now to the drawings, and more particularly to Figures 1-9, 
there are shown exemplary embodiments of the method and structures of the 
present invention. 

In contrast to conventional methods as referenced in the above 
referenced C. W. Wu et al paper, an exemplary embodiment of the present 
invention embeds and/or extracts an image (e.g., a watermark) using at least 
two other images. 

An exemplary embodiment of the present invention also differs from 
the conventional art of using conventional halftoning to embed a watermark 
into two images, as referenced in U.S. Patent Nos. 5,734,753 and 5,790,703 
and in M. Fu et al, "Data hiding in halftone image by stochastic error 
diffusion," Proc. IEEE Int. Conf. ASSP, pp. 1965-1968, 2001, M. Fu et a al, 
"A novel method to embed watermark in different halftone images: data 
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hiding by conjugate error diffusion (dhced)/' Proc. IEEE Int. Conf. ASSP, pp. 
III-529-532, 2003 and M. Fu et al., "A novel self-conjugate halftone image 
watermarking technique " Proc. IEEE Int. Symp. Circ. Syst., pp. III-790-793, 
2003, as an exemplary embodiment of the present invention utilizes a 
5 halftoning algorithm on the Cartesian product of pixel spaces, i.e., halftoning 
is done on n-tuples of pixels. In this manner, an exemplary embodiment of 
the present invention overcomes the shortcomings of the conventional 
methods described above. 

A description of one exemplary embodiment of the present invention 

10 follows. Given grayscale images Af, A2', and A3', two halftone images Ai 
and A2 may be constructed. The image A3' represents a watermark which is 
to be embedded into the two images Af, A2'. 

From the two images Ai and A2, a watermark image A3 can be 
extracted by the operation A3(i, j) = Ai(i, j) o A2(i, j). The binary operator o is 

1 5 OR in the case of overlaying images, but with the present invention can be 

any (not necessarily synunetric) binary operator such as Exclusive-OR, AND, 
etc., without limitation. A binary operation or operator means that the 
operation has two input argimients. Therefore, an exemplary embodiment of 
the present invention is not limited to any specific type of binary operation. 

20 An aspect of an exemplary embodiment of the present invention is to 

construct Ai and A2 such that they resemble Ai' and A2' respectively, and 
such that A3 resembles A3\ 

An exemplary embodiment of the present invention constructs images 
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Ai and A2 via digital half-toning on a Cartesian product of color spaces. 

The input to an exemplary digital half-toning method in accordance 
with the present invention are pixels of an image U = Ai' x A2' x A3' that is a 
composite of the three images Ai', A2' and A3' . The pixels of U are given 
by U(i, j) = (Ai ' (i, j), A2' (i, j), A3' (i, j)). In this exemplary embodiment the 
output halftone images Ai and A2 are binary, and, thus, the possible values for 
each pixel pair (Ai (i, j), A2 (i, j)) are (0, 0), (1, 0), (0, 1), and (1, 1). 

Thus, since A3 (i, j) = Ai (i, j) o A2 (i, j), a possible vector in the space 
of triples of pixels for Ai x A2 x A3 is one the foUov^ing: (0, 0, 0 o 0), (0, 1, 0 o 
1), (1, 0, 1 o 0), and (1, 1, 1 o 1) which will be called "output vectors." 

An exemplary digital half-toning algorithm may operate in a space of 
triples of pixels to halftone the images (Ai' x A2' x A3' ) using these 4 output 
vectors, and the resulting output halftone image (Ai x A2 x A3 ) is such that 
Ai a Ai', A2 » A2' and A3 « A3' , where Ai w Ai' indicates that Ai looks 
like Ai' when viewed at a distance. The desired output images are Ai and A2 
which, by construction, combine via an exemplary embodiment of the 
inventive watermark extraction operation to produce A3. 

There are several choices for a digital halftoning method that may be 
used in conjxmction with an exemplary embodiment of the present invention. 
The selection of the digital halftoning method may be determined by the 
desired tradeoff between processing speed and the quality of the halftone 
generated. 

For example, a vector error diffusion method (VED), as described in 
YOR920030609US1 



H. Haneishi et al, "Color digital halftoning taking colorimetric color 
reproduction into account", Journal of Electronic Imaging, vol. 5, pp. 97-106, 
1996, is a one-pass method that is fast and produces good halftones. Another 
example is a modified error diffiision method described in R. Adler et al, 
5 "Error bounds for error diffusion and other mathematical problems arising in 
digital halftoning," Proc, of SPIE, vol. 3963, pp. 437-443, 2000 and in R. 
Adler et al, "Error bounds for error diffusion and related digital halftoning 
algorithms," Proc. IEEE Int. Symp. Circ. Syst., vol. II, pp. 513-516, 2001, 
which alleviates some of the problems that may be due to large errors in the 
1 0 vector error diffusion method. 

These one-pass methods analyze and process each n-tuple of pixels 
once in a specific order. This ordering of pixels to be processed, and the 
causality of the error diffusion filter (such as the Jarvis filter or the Stucki 
filter) may cause anisotropic artifacts in the halftone image. For more 
1 5 information on error diffusion, see the books "Digital Halftoning" by R. 

Ulichney, MIT Press, 1987 and "Digital Color Halftoning" by H. Kang, SPIE 
Press monograph vol. PM68, 1999. 

Further, in some digital watermarking or data hiding applications there 
could be constraints on the relationships between pixels in an image that are 
20 far apart and, for these applications, error diffusion may not be appropriate. 
An example of such a watermarking application is described below. 

Therefore, an exemplary embodiment of the present invention uses an 
iterative isotropic halftoning method in order to embed the images. This 
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halftoning method is described in C. W. Wu, "Multimedia Data Hiding and 
Authentication via Halftoning and Coordinate Projection", EURASIP Journal 
on Applied Signal Processing, vol. 2002, no. 2, pp. 143-151, 2002, vs^here it 
was used to embed a single image inside another image. In contrast, the 
present invention embeds one or more images into two or more images. 
An exemplary pseudo code for this halftoning method follows: 

for each iteration /* Loop through iterations */ 

for each i /* Loop through each row */ 

for each j /* Loop through each column */ 



=argmin Error (o). 

oeP 

setOutimage(i,j) = Omin. 
endfor (j) 
endfor (i) 

if Outimage has not changed between two iterations or maximum number 
of iterations reached, exit iterations loop. 
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for each output vector o = (oi, 02, 03) € P 



/*Loop through all possible output vectors */ 



replace Outimagek(iJ) with Ok for k = 1, 2, 3. 




endfor 



find output vector Omin e P that minimizes Error, i.e. 
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endfor 
Where: 

Ai', A2' and A3' are input images; 
5 P is a set of output vectors; 

Output: set (Ai,A2,A3) = Outimage which resembles (Ai', Ai, A3'). 
The two halftone images are Ai and A2. 

Vk determines how strongly the error in each image is minimized; and 
L is a linear space-invariant model of the human vision system. 

10 

OutimagCk denotes the k-th component of Outimage, i.e. if Outimage = 
(Ai, A2, A3) , then OutimagCk = Ak. The input are three images (Ai', A2', A3') 
and the output are 2 halftone images Ai and A2. The constant Vk determines 
how strongly the error in each corresponding image is minimized. 

15 In an exemplary embodiment of the present invention, for each 

iteration, the algorithm loops through each pixel of A\ x A2' x A3' and 
selects the output vector from P that when put in the corresponding position in 
Outimage minimizes the error measure "Error". The pixel in Outimage at that 
position is then set as this output vector. 

20 The error measure "Error" may be calculated as follows: for each of 

the images, the norm of the low pass filtered version of the difference between 
Ak' and OutimagCk is calculated and the weighted sum of these norms form 
the error measure. Thus, minimizing this error measure allows the low pass 

YOR920030609US1 



14 



filtered version of the components of Outimage to resemble each Ak'. 
The low pass filter (expressed as the linear operator L) may be taken firom a 
HVS filter. Some examples of such filters can be found in R. Nasanen, 
"Visibility of halftone dot textures", IEEE Trans. Syst. Man, Cybernetics, vol. 
14, no. 6, pp. 920-924, 1984 and in J. Sullivan et al, "Design of minimum 
visual modulation halftone patterns," IEEE Trans. Syst. Man, Cybernetics, 
vol. 21, no. 1, pp. 33-38, 1991. 

In an exemplary embodiment of the present invention, the 5 by 5 filter 
shown below may be used. The filter is given in K. R. Crounse, "Image 
halftoning with cellular neural networks", IEEE Trans. Circ. Syst-II, vol. 40, 
no. 4, pp. 267-283, 1993, normalized to have the sum of the coefficients equal 
tol: 



0.00492 
0.01391 
0.02100 
0.01391 
0.00492 



0.01391 
0.05810 
0.09772 
0.05810 
0.01391 



0.02100 
0.09772 
0.01618 
0.09772 
0.02100 



0.01391 
0.05810 
0.09772 
0.05810 
0.01391 



0.00492 
0.01391 
0.02100 
0.01391 
0.00492 



In this exemplary embodiment, the filter L may be chosen to be the 
same for each of the images Ai', A2' and A3'. In some applications, for 
example, when the original images and the watermark images are intended to 
be viewed at different distances, the filter L can be different for each image. 

In accordance with an exemplary embodiment of the present invetnion, 
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to extract the watermark from the two halftone images Ai and A2, the binary 
operator o may be applied to each pair of pixels (Ai(ij), A2(i j)) resulting in 
the extracted watermark image W. In other words, the (ij)-th pixel of W is 
given by W(iJ) = Ai(iJ) o A2(ij). 

Because of the linearity, space-invariance, and a relatively small 
support of L, the computation of the variable "Error" in the iterative isotropic 
halftoning method may be sped up by updating the change in the variable 
"Error" due to changing a single pixel of "Outimage." In other words, 
changing a single pixel of "Outimage" only changes a small number (on the 
order of the size of the support of L) of entries of L(Outimagek - Ak'). This 
technique has been used in dither mask construction when the output is binary 
(see e.g. C. W. Wu, G. Thompson and M. Stanich, "A unified framework for 
digital halftoning and dither mask construction: variations on a theme and 
implementation issues," Proc, IS&T's NIP 19: Int. Conf. on Digital Printing 
Tech., pp. 793-796, 2003), but is equally usefiil in an exemplary embodiment 
of the present invention. 

The output image "Outimage" may be initialized with an image having 
pixels having random values from a set of output vectors or from a uniform 
image of a single output vector. To reduce the number of iterations, 
"Outimage" may also be initialized by performing vector (or modified) error 
diffiision and using the output from the vector (or modified) error diffusion as 
the initial "Outimage." 
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GAMUT MAPPING 

For the halftone image to resemble the original image, the error in the 
digital halftoning algorithm should be small or at least bounded for arbitrarily 
large images. In order for the error in the error diffusion halftoning method to 
5 be bounded, it was shown in the references by Adler et al. cited above that the 
pixels of the input image (Ai' x A2' x A3' ) should be within a convex hull of 
the output vectors. This analysis is also applicable to general halftoning 
algorithms, and the low pass behavior of the human vision system can be 
viewed as an averaging behavior and the original image is approximated by a 
10 convex sum of output pixels. In order to satisfy this convex hull condition, an 
exemplary embodiment of this invention uses scaling or gamut mapping of the 
images. 

In an exemplary embodiment of the present invention, the gamut 
mapping is as follows: Let S be the set of 3-tuples of pixels in (Ar x A2' x 
15 A3' ), i.e. S = {(Af (i, j), A2' (i, j), A3' (i, j))}. Furthermore, let Si, S2, and S3 
be the sets of pixels of images A\\ Ai, and A3 'respectively, i.e. Si = {Ai' (i, 
j)}, S2 = {A2' (ij)}, and S3 = {A3' (ij)}. For simplicity, consider only a 
gamut mapping M that maps a pixel p into a pixel M(p) of the following form: 

20 Forp = (pi,p2,p3)€S, (1) 

M(p) = (Sipi + di, S2P2 + d2, S3P3 + d3) 

where: 
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Si are real numbers denoting scaling factors; and 
dj are offset vectors in the color space. 



Let H be the (closed) convex hull of the output vectors. Then H can be 



5 expressed by a set of linear inequalities: H = {x : Ax < b}. A conunonly used 
algorithm for finding the convex hull of a set of points is the QhuU algorithm 
described in C. B. Barber et al, "The quickhuU algorithm for convex hulls," 
ACM Trans. Math. Software, vol. 22, no. 4, pp. 469-483, 1996. An 
optimization problem used to find the (optimal) gamut mapping may be 
1 0 formulated as follows : 



1 5 problem will be used as the gamut mapping M, i.e. every pixel of the input 
composite image Af x A2' x A3' is scaled by M before they are halftoned by 
the exemplary halftoning algorithm in accordance with the present invention. 

Other forms of the objective fimction may be used for other exemplary 
embodiments of the present invention. For example: 



max mm 

Si A 




under the constraint that M(S) e H 



(2) 



a, 



The set of parameters {Si, dj} which solves the above optimization 



20 



max 5,52*3 under the constraint that M(S) e H 



(3) 
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The coefficients a\ in Equation (2) detennine, relatively, the "penalty" 
of scaling each image. The smaller each a\ is, the more the corresponding 
image will be scaled. For instance, in a watermarking application, the two 
images Ai and A2 should retain most of the fidelity of the original images, 
5 while in contrast the watermark image, which in many applications is assumed 
to be a less complex image, may accept more distortion due to scaling. Thus, 
in this case, as may be smaller than ai and a2. 

It is clear that the constraints in Eq. (2) are linear, i.e. the inequality 
constraints M(S) € H is linear in the variables Si, di. This is true since M(S) e 
10 H for each pixel p = (pi, p2, ps) e S can be written as: 

Z>,(^.A+^J^6 (4) 

Where: 

15 A = [Ai : A2 : A3] is decomposed as a concatenation of the matrices 

Ai A2 and A3 and A, b are the matrices describing the convex hull H = 
{x:Ax<b}. 

If the number of pixels in the images is large, the constraints M(S) e H 
20 can be time consuming to compute. One way to speed up the computation is 
to not use all the pixels to compute the gamut, i.e. replace S with a subset 
^''c 5 . This results in a larger feasible region as the set of parameters (Si,di) 
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satisfying the constraint M(S") 6 H (i.e. the set {(si,di):M(S") e H }) is larger 
than the set of parameters satisfying M(S) e H . This meEns thdt the convex 
hull condition may not be strictly satisfied. However, if S" is close to S, then 
the gamut mapping obtained using S" is close to the gamut mapping obtained 
using S. In other words, the convex hull condition violation is mild. 
In experiments by the inventor, S" being 5 percent of the pixels of S still gave 
good results. 

Another way to speed up the computation is the following 
simplification which produces a smaller feasible region for the optimization 
problem in Eq. (2), and is easier to compute than M(S) e H. However, since 
the feasible region is smaller, the gamut mapping is also less optimal. 

Let Vi be the set of extreme points of the convex hull of Si. This 
means that the convex hull of Vj is equal to the convex hull of Si, Next 
replace the constraint M(S) e H with M(S') e H, where S' = {(pi, p2, ps) : Pi e 
Vi}. The convex hull of S' is then larger than the convex hull of S, i.e. the 
feasible region {(si,di):M(S') e H} is smaller than {(si,di):M(S) e H}. 

The gamut mapping may be computed using the pixel values of the 
specific images Ai', A2' and A3'. This gamut mapping in general is not 
appropriate for another set of images. By replacing Si, S2, and S3 with the 
extreme points (which by abuse of notation denoted herein as Vi, V2, and V3, 
respectively) of the possible gamut (i.e. the possible range of the pixel values) 
of the images Ai', A2' and A3', respectively, a gamut mapping obtained using 
the resulting M(S') e H can be used for any possible image A\\ A2' and A3'. 
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For instance, suppose that the pixel in the image A2' can take on 
values between 0 and 1 . Then by replacing S2 with the set {0, 1 } (which is the 
set of extreme points of the interval [0,1]) and solving the optimization 
problem using the constraints M(S') e H where S' = {(Af (i, j), p, A3'(i, j)) : 
5 p e {0,1 } }, the resulting gamut mapping can be used with images Ai' and 
A3 'and any image A2' whose pixels lie in the interval [0, 1]. If all three Si's 
are replaced with the set {0, 1 }, and the OR operation is used for extracting 
the watermark, i.e. the output vectors are (0, 0, 0), (0, 1, 1), (1, 0, 1), and (1, 1, 
1), then the following gamut mapping: si = S2 = S3 = 0.25, di = d2 = 0.5, and d3 
10 == 0.75 is obtained, which can be used for all grayscale images Ai A2' and 
A3'. Thus, by restricting the feasible region this way, sub-optimal gamut 
mappings that cause more distortion to the original images are obtained, but 
they are applicable to a larger set of images. 

In accordance with an exemplary embodiment of the invention, the 
15 error calculation in the digital halftoning algorithm can also be weighted 

according to the desired fidelity. For instance, if a low fidelity watermark is 
adequate, the error calculation can be skewed to put more emphasis in 
reducing the error in the two main images. This is reflected in the parameters 
Vi in the exemplary pseudo code shown above. In this case V3 would be 
20 smaller than vi and V2. 

In an exemplary embodiment of the present invention, the condition 
that the input pixels lie in the convex hull of the output vectors may be 
violated mildly without much loss in the quality of the halftone images. This 
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allows the gamut mapping to distort the image less. 
EXAMPLES 

Figures 1 A, IB, and IC illustrate the results of an exemplary 
5 embedding process in accordance with the present invention of embedding an 
image entitled "Lena" shown in Fig. IC into two other images (i.e. those 
shown in Figures 1 A and IB). The first image shown in Fig. 1 A is entitled 
"Baboon" and the second image shown in Fig. IB is entitled "Boat." The 
gamut mapping was obtamed using ai = 1, = 1, and as = 0.5. Figure IC is 
1 0 the extracted watermark image obtained by performing the OR operation on 
the pixels of the images of Figs. 1 A and IB. 

Figures 2A, 2B, and 2C illustrate the results of an embedding process 
in an exemplary embodiment in accordance with the present invention as 
shown in Figs. 1 A, IB, and IC, except that the "Lena" image shown in Fig. 
15 2C is embedded into two images that are uniform gray images (Figs. 2A and 
2B). The gamut mapping here was computed using ai = = aj = 1 . 

Figures 3 A, 3B, and 3C illustrate the results of yet another exemplary 
embedding of the "Lena" image of Fig. 3C into the "Baboon" image of Fig. 
3 A and the "Baboon" image of Fig. 3B where the binary operation is the 
20 Exclusive-OR operation in accordance with the present invention. Note that 
the two "Baboon" images (Figs. 3 A and 3B) appear to be the same, but the 
halftone dots are arranged differently so that when an Exclusive-OR is 
applied, the watermark image ("Lena") of Fig. 3C emerges. The gamut 
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mapping here was computed using ai = a2 = as = 1. 

Note that the extracted watermark images in these examples are of 
high fidelity. This is in contrast with the conventional methods, where a 
residual of the two halftone images is evident in the extracted watermark 
5 image. 

In yet another exemplary embodiment of the present invention, as 
shown in Figs. 4A - 4C, two images may be generated from reoriented 
versions of the same image. For example, the same image may be rotated 180 
degrees to create a second image. These two images then form the images 

10 Ai', and Ai'. The watermark image A3' is then embedded into these images 
to create the halftone images Ai, A2 where A2 is Ai rotated 180 degrees. The 
idea is that the watermark can be extracted from a single image Ai and its 
reoriented version A2. In other words the image Ai contains all the 
information needed to extract the watermark. Note that for this exemplary 

15 embodiment, there is an acausal relationship between the pixels of Ai and A2 
and, thus, error diffusion may not be appropriate in this setting. 

For this exemplary embodiment, the pseudo code described above may 
be modified as follows: For an R-pixels by C-pixels image, when computing 
"Error(o)", in addition to replacing Outimagek(i, j) with Ok, Outimagei(R - 

20 i+l,C - j+1) may be replaced with 02, Outimage2(R - i+l,C - j+1) may be 
replaced with Oi, and Outimage3(R - i+l,C - j+1) may be replaced with 
O2 o a, . Note that the (R-i+l,C-jH-l)-th pixel is the (ij)-th pixel rotated 180 

degrees. Then when o^^ = (o[ , , O3 ) is found, set Outimage(i j) = Omin, and 
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set Outimage(R-i+l ,C-j+l) = (Oj , o^Oj <> oj ) , Assuming that the binary 
operation o is symmetric, then the watermark image A3' must be such that it 
is invariant under 1 80 degree rotation. The gamut mapping may be chosen 
using ai = with the additional constraints that di = d2, si = S2. Again, an 
5 example of this is shown in Figures 4A - 4C where Figure 4A is the halftone 
image "Lena" and Figure 4A is the watermark that is revealed when the OR 
operation is applied to Figure 4A and a copy of itself that is rotated 180 
degrees (Figure 4B). The gamut mapping is obtained using ai = a2 = 1, as = 
0.7. 

10 In general, an exemplary embodiment of the present invention may be 

used for more complicated applications of image watermarking, data hiding, 
and viewing images under different viewing conditions. For an image A that 
is represented as a matrix of pixels the (k, l)-th pixel of A may be denoted as 
A(k, 1). Each pixel may be a vector in a color space T (which may vary from 

15 image to image), i.e. A(k, 1) g T. Given n images Ai, . . An, an exemplary 
watermark extraction transform <1> = (cpi, . . . , (pm) constructs m images as 
follows: O (Ai, . . . , An) = (Bi, . . . , Bm) where Bj = (pj(Ai, . . . , An), i.e. Bj are 
images created from the sets of images Ai, . . . , A„. The transform (pj operates 
pixel wise on the images Ai, . . . , An, i.e. Bj(k, 1) = (pj(Ai(k, 1), . . . , An(k, 1)). 

20 Given a set of n + m images A' 1, . . . , A'n, and B' 1, . . . , B'm, an 

exemplary embodiment of the present invention may create n images (Ai, . . . 
, An) that resemble A'l, . . . , A'n such that the m images Bi, . . . , Bm that are 



YOR920030609US1 



24 

extracted from (Ai, . . . , An) using transform O may resemble B'l, . . . , B'm. 

This exemplary embodiment uses a digital halftoning algorithm. 
Digital halftoning can be described in the following general form: given an 
image I a digital halftoning process creates a halftone image T such that each 
pixel of r is a member of a restricted set of output pixels O and, fiirthermore, 
r resembles I under some metric d, i.e. d(I, V ) is small. 

This exemplary embodiment of the present invention specifies the set 
of output pixels in order to use a digital halftoning algorithm. The pixels of Ai 
are chosen from the set Oi. Let R c Oi x . . . x On = {(pi, . . . , Pn) : Pi ^ Oi} 
be the possible output vectors for the combined image Ai x . . . x An. The 
subset R can be a strictly smaller subset of Oi x . . . x On in order to express 
additional relationships between pixels in different images. From R the set of 
extended output vectors P = {(p, (pi(p), . . . , cpm(p)) : p e R} is formed and 
(A'l X . . . X A'n X B'l X . . . x B'm) may be halftoned using P as the set of 
possible output vectors. 

An exemplary pseudocode of the iterative isotropic halftoning 
algorithm for this general case is as follows: 

for each iteration /* Loop through iterations */ 

for each i /* Loop through each row */ 

for each j /* Loop through each column */ 

for each output vector o = (oi, . . On+m ) e P 
/*Loop through all possible output vectors */ 
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replace Outimagek(i, j) with Ok for k = 1 , . . . , n+m. 
set 

endfor 

5 find output vector Omin 6 P that minimizes Error, i.e. 

Otnm ^^xgmin Error (o). 

set Outimage(i, j) = Omin- 
endfor (j) 
endfor (i) 

1 0 if Outimage has not changed between two iterations or maximum number 

of iterations reached, exit iterations loop, 
endfor 

Where: 

15 Ai%..., An', Bi',..., Bm' are the input images; 

P is a set of output vectors; 

Output: set (Ai,...,An, Bi, ...^ Bm )= Outimage which resembles 
(Af,...,An\Bi\„., Bm'). The output halftone images are Ai,...,An; and 
and vf determines how strongly the error in each image is 

20 minimized. 

To satisfy the convex hull condition, the pixels of (A'l x . . . x A'n x 
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B'l X . . . X B'm) are first scaled via the gamut mapping M. The gamut 
mapping M is calculated by solving the optimization problem 



max min— 



(5) 



under the constraint that (^,/7i + 1/, , . . . , s„^„p„^„ + d^^^ ) € // for all pixels 
/7 = (/7p...,j7„^.„)of (A'l X ... X A'n X B'l X . . , X B'm) where H is the 
convex hull of the output vectors in P. The gamut mapping is then given by: 

In another exemplary embodunent, the objective function in Equation 
(5) may be replaced by: 



With the above in mind, an exemplary method of the invention may be 
generalized with reference to the flowchart shown in Fig. 6. The flowchart 
starts at step 600 and continues to step 602 where the control routine inputs 
images. For example, the control routine may input images Ai An', and 
Bi',...,Bm' at step 602. 

The control routine may then continue to step 604, where gamut 



maxn.y^ 



(6) 
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mapping, as described above, is applied to the input images. The control 
routine may then continue to step 606 where the digital halftoning method 
described above is applied to the gamut mapped images. Then, at step 608, 
the control routine ends and may, optionally, retum to the process that called 
the control routine of Fig. 6. 

In another exemplary embodiment of the present invention different 
combinations of images produce different watermarks. For example, for the 
case (n=3, m=2), given three images Ai, A2, and A3, combining Ai and A2 
may output one watermark Bi, and combining Ai and A3 may output another 
watermark B2. 

An example of the results of using this exemplary embodiment is 
shown in Figures 5A - 5C where the gamut mapping was computed using ai 
= a2 = a3 = 1, and a4 = as = 1/2. Figures 5 A, 5B, and 5C show halftone 
images of a "Baboon," a "Boat," and a "Boat on a lake," respectively. The 
"Peppers" image shown in Figure 5D is obtained by overlaying the images of 
Figure 5A and Figure 5B, and the "Lena" image shown in Figure 5E is 
obtained by overlaying the images of Figure 5 A and Figure 5C. 

MULTIBIT IMAGES, COLOR IMAGES AND OTHER EXTENSIONS 

Yet another exemplary embodiment of the present invention may also 
provide a multi-bit output. In this case, gray levels of two images are simply 
added (with clamping) to produce the gray level of an overlaid image. When 
each component of the output vector is from a q-bit quantizer (e.g. the binary 
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case is when q = 1), the number of output vectors is large when q is large. In 
this case, the innermost loop of the exemplary pseudo code described earlier 
that searches for the output vector that mimmizes "Error" may take many 
computations. Since Omin is the output vector that gives the minimal value of 
"Error" among all choices of the output vectors at location (i, j), the 
computation of = argmin Error (o) is a discrete optimization problem. 

For large q this may be relaxed to a continuous optimization problem and 
solved using nonlinear programming and the result may then be quantized by 
a q-bit quantizer. 

This exemplary embodiment may also be used for color images. In 
this case each pixel in a color image lies in a multi-dimensional color space 
such as CIELab, RGB, CMYK, or the like, and digital halftoning may be 
performed in the Cartesian product of these color spaces. Instead of 
processing in the Cartesian product of multi-dimensional color spaces, the 
images may also be decomposed into their color planes and each plane may be 
processed independently as a grayscale image and the results may then be 
combined afterwards. This altemative approach appears to work well in the 
RGB color space. 

Similarly, since the human aural system exhibits a low pass behavior, 
similar to the human visual system, an exemplary embodiment of the present 
invention may be used to watermark audio data and/or signals. 

Figure 7 illustrates a typical hardware configuration of an information 
handling/computer system for use with the invention and which preferably has 
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at least one processor or central processing unit (CPU) 711. 

The CPUs 71 1 are interconnected via a system bus 712 to a random 
access memory (RAM) 714, read-only memory (ROM) 716, input/output 
(I/O) adapter 718 (for connecting peripheral devices such as disk units 721 
5 and tape drives 740 to the bus 712), user interface adapter 722 (for connecting 
a keyboard 724, mouse 726, speaker 728, microphone 732, and/or other user 
interface device to the bus 712), a communication adapter 734 for connecting 
an information handling system to a data processing network, the Internet, an 
Intranet, a personal area network (PAN), etc., and a display adapter 736 for 

10 connecting the bus 712 to a display device 738 and/or printer 740. 

In addition to the hardware/software environment described above, a 
different aspect of the invention includes a computer-implemented method for 
performing the above method. As an example, this method may be 
implemented in the particular environment discussed above. 

15 Such a method may be implemented, for example, by operating a 

computer, as embodied by a digital data processing apparatus, to execute a 
sequence of machine-readable instructions. These instructions may reside in 
various types of signal-bearing media. 

This signal-bearing media may include, for example, a RAM 

20 contained within the CPU 71 1 , as represented by the fast-access storage for 
example. Alternatively, the instructions may be contained in another 
signal-bearing media, such as a magnetic data storage diskette 800 (Figure 8), 
directly or indirectly accessible by the CPU 711. 
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Whether contained in the diskette 800, the computer/CPU 71 1, or 
elsewhere, the instructions may be stored on a variety of machine-readable 
data storage media, such as DASD storage (e.g., a conventional "hard drive" 
or a RAID array), magnetic tape, electronic read-only memory (e.g., ROM, 
5 EPROM, or EEPROM), an optical storage device (e.g. CD-ROM, WORM, 
DVD, digital optical tape, etc.), paper "punch" cards, or other suitable 
signal-bearing media including transmission media such as digital and analog 
and communication links and wireless. In an illustrative embodiment of the 
invention, the machine-readable instructions may comprise software object 

10 code, compiled from a language such as "C", etc. 

Figure 9 illustrates yet another exemplary embodiment of a system 900 
for embedding an image into two other images in accordance with the present 
invention. The system 900 includes an image input device 902, a gamut 
mapping device 904 and a digital halftoning device 906. The image input 

15 device 902 inputs the image to be embedded as well as the two other images 
in which the one image will be embedded. The gamut mapping device 904 
receives the images input by the image input device 902 and performs the 
gamut mapping process described above on the images. The digital halftoning 
device 906 performs a digital halftoning process on a Cartesian product of 

20 color spaces to embed the image into the two images. 

While the invention has been described in terms of several exemplary 
embodiments, those skilled in the art will recognize that the invention can be 
practiced with modification. 
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Further, it is noted that. Applicants' intent is to encompass equivalents 
of all claim elements, even if amended later during prosecution. 
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